Electronic device for displaying a print preview

ABSTRACT

An electronic device in the present disclosure includes the following: an imaging device configured to scan a paper sheet, wherein the paper sheet comprises a first side and a second side, wherein scanning the paper sheet produces an image; a display device; a printing image acquisition unit configured to acquire a printing image; an image forming apparatus configured to print the printing image; and a display control unit configured to control the display device to display, if the produced image includes an image of a paper sheet, a composite image that combines the acquired printing image and the image of the paper sheet within the produced image.

REFERENCE TO RELATED APPLICATIONS

This application is a divisional application of, claims priority to, andincorporates herein by reference the contents of U.S. patent applicationSer. No. 14/264,583, filed in the United States Patent and TrademarkOffice on Apr. 29, 2014, which is based upon and claims priority to thecorresponding Japanese Patent Application No. 2013-095360, filed in theJapan Patent Office on Apr. 30, 2013, and Japanese Patent ApplicationNo. 2013-136142, filed in the Japan Patent Office on Jun. 28, 2013, theentire contents of which are incorporated herein by reference.

FIELD

The present disclosure relates to an electronic device that displays acomposite image, which combines an image scanned by an imaging devicewith additional information, on a display device as a print preview.

BACKGROUND

In a typical print preview, before an image to be printed by an imageforming apparatus is actually printed on a paper sheet, a preview of theimage to be printed on the paper sheet is artificially displayed.

In the typical print preview, an appearance of an artificially displayedpaper sheet may differ from an appearance of an actual paper sheet when,for example, a color of the artificially displayed paper sheet is afixed color, such as white, that is different from a color of the actualpaper sheet. Accordingly, there is a fear in that an artificial displayof the preview of the image to be printed on a paper sheet may differfrom the resulting printed image obtained by actually printing the imageon a paper sheet.

SUMMARY

An electronic device according to an embodiment of the presentdisclosure includes: an imaging device; a display device; a printingimage acquisition unit; an image forming apparatus; and a displaycontrol unit. The imaging device is configured to scan a paper sheet,wherein the paper sheet comprises a first side and a second side,wherein scanning the paper sheet produces an image. The printing imageacquisition unit is configured to acquire a printing image. The imageforming apparatus is configured to print the printing image. The displaycontrol unit is configured to control the display device to display, ifthe produced image includes an image of a paper sheet, a composite imagethat combines the acquired printing image and the image of the papersheet within the produced image.

A print preview method according to one embodiment of the presentdisclosure includes: (i) scanning, via an imaging device, a paper sheet,wherein the paper sheet comprises a first side and a second side,wherein scanning the paper sheet produces an image; (ii) acquiring, viaa printing image acquisition unit, a printing image; (iii) printing, viaan image forming apparatus, the printing image; and (iv) controlling,via a display control unit, a display device to display, if the producedimage includes an image of a paper sheet, a composite image in which theacquired printing image and the image of the paper sheet included in theproduced image are combined.

A non-transitory computer-readable recording medium according to oneembodiment of the present disclosure has stored a print preview programexecutable by a computer of an electronic device. The print previewprogram includes: a first program code for causing the computer to scana paper sheet, wherein the paper sheet comprises a first side and asecond side, wherein scanning the paper sheet produces an image; asecond program code for causing the computer to acquire a printingimage; a third program code for causing the computer to print theprinting image; and a fourth program code for causing the computer tocontrol a display device to display, if the produced image includes animage of a paper sheet, a composite image in which the acquired printingimage and an image of the paper sheet included in the produced image arecombined.

Additional features and advantages are described herein, and will beapparent from the following detailed description and the figures.

BRIEF DESCRIPTION OF THE DRAWINGS

All drawings are intended to illustrate some aspects and examples of thepresent disclosure. The drawings described are only schematic and arenon-limiting, and are not necessarily drawn to scale.

FIG. 1 illustrates a configuration of an image forming system accordingto an embodiment of the present disclosure.

FIG. 2 illustrates a configuration of an MFP illustrated in FIG. 1.

FIG. 3 illustrates a configuration of a mobile device illustrated inFIG. 1.

FIG. 4 illustrates an operation of the mobile device illustrated in FIG.3 performed when a preview image is displayed.

FIG. 5 illustrates an example of a head position of a paper sheet withina scanned image displayed on a display unit illustrated in FIG. 3.

FIG. 6 illustrates an example of a preview image displayed on thedisplay unit illustrated in FIG. 3.

FIG. 7 illustrates another example of the preview image displayed on thedisplay unit illustrated in FIG. 3.

FIG. 8 illustrates another example of the preview image displayed on thedisplay unit illustrated in FIG. 3.

FIG. 9 illustrates another example of the preview image displayed on thedisplay unit illustrated in FIG. 3.

FIG. 10 illustrates another example of the preview image displayed onthe display unit illustrated in FIG. 3.

FIG. 11 illustrates a configuration of an image forming system accordingto another embodiment of the present disclosure.

FIG. 12 illustrates a configuration of a mobile device illustrated inFIG. 11.

FIG. 13 illustrates an operation of the mobile device illustrated inFIG. 12 performed when a binding orientation is set for double-sidedprinting performed on the MFP.

FIG. 14 illustrates an operation continued from FIG. 13.

FIG. 15 illustrates another operation continued from FIG. 13.

FIG. 16 illustrates composite image displaying processing in theoperations illustrated in FIG. 13 to FIG. 15.

FIG. 17 illustrates an example of a preview image displayed on a displayunit illustrated in FIG. 12.

FIG. 18 illustrates another example of the preview image displayed onthe display unit illustrated in FIG. 12.

FIG. 19 illustrates another example of the preview image displayed onthe display unit illustrated in FIG. 12.

FIG. 20A illustrates an example of the preview image displayed on thedisplay unit when the binding orientation is set to “left bind”.

FIG. 20B illustrates an example of the preview image displayed on thedisplay unit when the binding orientation is set to “top bind”.

DETAILED DESCRIPTION

Various embodiments are described below with reference to the figures.It should be understood, however, that numerous variations from thedepicted arrangements and functions are possible while remaining withinthe scope and spirit of the claims. For instance, one or more elementsmay be added, removed, combined, distributed, substituted,re-positioned, re-ordered, and/or otherwise changed. Further, where thisdescription refers to one or more functions being implemented on and/orby one or more devices, one or more machines, and/or one or morenetworks, it should be understood that one or more of such entitiescould carry out one or more of such functions by themselves or incooperation, and may do so by application of any suitable combination ofhardware, firmware, and/or software. For instance, one or moreprocessors may execute one or more sets of programming instructions asat least part of carrying out one or more of the functions describedherein.

An embodiment of the present disclosure is described with reference tothe accompanying drawings.

FIG. 1 illustrates a configuration of an image forming system 10according to the embodiment.

As illustrated in FIG. 1, the image forming system 10 includes aplurality of multifunction peripherals (MFPs) including an MFP 20, amanagement server 30 such as a personal computer (PC) that manages theplurality of MFPs, and a mobile device 40 such as a smartphone. Theplurality of MFPs, the management server 30, and the mobile device 40 ofthe image forming system 10 are communicably connected to one anotherthrough a network 11 such as a local area network (LAN) or the Internet.The mobile device 40 constitutes an electronic device according to thepresent disclosure.

The configuration of each of the MFPs of the image forming system 10 isthe same as the configuration of the MFP 20. Therefore, the MFP 20 isdescribed below as a representative of the plurality of MFPs of theimage forming system 10.

FIG. 2 illustrates the configuration of the MFP 20.

As illustrated in FIG. 2, the MFP 20 includes an operation unit 21serving as an input device such as buttons through which variousoperations are input by a user, a display unit 22 serving as a displaydevice such as a liquid crystal display (LCD) that displays variouskinds of information, a scanner 23 that reads an image from an originaldocument, a printer 24 that executes printing on a paper sheet such asprinting paper, a FAX communication unit 25 that performs FAXcommunications to and/or from an external facsimile machine (not shown)through a communication line such as a public telephone line, a networkcommunication unit 26 that communicates to and/or from an externalapparatus through the network 11 illustrated in FIG. 1, a storage unit27 including a nonvolatile storage device such as an electricallyerasable programmable read only memory (EEPROM) or a hard disk drive(HDD) that stores various kinds of data, and a control unit 28 thatcontrols the entire MFP 20.

The operation unit 21 may include an input device including a touchpanel together with the display unit 22.

The control unit 28 includes, for example, a central processing unit(CPU), a read only memory (ROM) that stores programs and various kindsof data, and a random access memory (RAM) used as a work area for theCPU. The CPU executes a program stored in the ROM or the storage unit27.

FIG. 3 illustrates a configuration of the mobile device 40.

As illustrated in FIG. 3, the mobile device 40 includes an operationunit 41 serving as an input device with, for example, buttons throughwhich various operations are input, a display unit 42 serving as adisplay device such as an LCD that displays various kinds ofinformation, an imaging unit 43 serving as an imaging device thatphotographs a still image or a moving image, a network communicationunit 44 that communicates to and/or from an external apparatus throughthe network 11 illustrated in FIG. 1, a storage unit 45 including anonvolatile storage device such as an HDD that stores programs andvarious kinds of data, and a control unit 46 that controls the entiremobile device 40.

The operation unit 41 may include an input device including a touchpanel together with the display unit 42.

The storage unit 45 stores a print preview program 45 a executed by thecontrol unit 46.

The control unit 46 includes, for example, a CPU, a ROM that storesprograms and various kinds of data, and a RAM used as a work area forthe CPU. The CPU executes a program stored in the ROM or the storageunit 45.

The control unit 46 executes the print preview program 45 a stored inthe storage unit 45, to thereby function as a printing image acquisitionunit 46 a, a display control unit 46 b, and an apparatus designationreception unit 46 c. The printing image acquisition unit 46 a acquires aprinting image to be printed on the paper sheet by the MFP. The displaycontrol unit 46 b controls the display unit 42 to display, when a papersheet is included in an image scanned by the imaging unit 43, acomposite image that combines the printing image acquired by theprinting image acquisition unit 46 a on the paper sheet within thescanned image. The apparatus designation reception unit 46 c receives adesignation of the MFP to be used for printing the printing image.

Next, a description is made of an operation of the image forming system10.

First, a description is made of the operation of the image formingsystem 10 performed when the printing image is generated by reading animage from an original document.

In response to an instruction issued through the operation unit 21, thecontrol unit 28 of the MFP 20 reads an image from an original documentby the scanner 23 to generate a printing image, and transmits thegenerated printing image to the management server 30 through the networkcommunication unit 26.

Therefore, the management server 30 receives and stores the printingimage transmitted from the MFP 20.

Next, a description is made of an operation of the mobile device 40performed when a preview image is displayed.

The user of the mobile device 40 may activate the print preview program45 a when it is desired to display the preview image on the mobiledevice 40, to thereby cause the control unit 46 of the mobile device 40to execute an operation illustrated in FIG. 4.

FIG. 4 illustrates the operation of the mobile device 40 performed whenthe preview image is displayed.

As illustrated in FIG. 4, the printing image acquisition unit 46 a ofthe mobile device 40 acquires the printing image to be printed on thepaper sheet by the MFP through the network communication unit 44 (StepS101). The printing image acquisition unit 46 a may acquire from themanagement server 30 the printing image generated from the originaldocument by the MFP and stored by the management server 30, or mayacquire a printing image other than the printing image generated fromthe original document by the MFP and stored by the management server 30.For example, the printing image acquisition unit 46 a may acquire aprinting image generated from the original document by the MFP 20 andtransmitted directly from the MFP 20 to the mobile device 40, or mayacquire a printing image generated by a PC (not shown) and transmittedto the mobile device 40.

Subsequently, the apparatus designation reception unit 46 c of themobile device 40 displays on the display unit 42 an MFP designationscreen used for receiving a designation of the MFP to be used for printthe printing image, to thereby receive the designation of the MFPthrough the operation unit 41 (Step S102).

Subsequently, the display control unit 46 b of the mobile device 40displays the image scanned by the imaging unit 43 on the display unit 42(Step S103). Note that an image that is “scanned” may be photographed,scanned, imaged, picked up, or otherwise read in by any suitablescanner, imaging device, imaging unit, or the like.

Subsequently, the display control unit 46 b determines whether or notthe paper sheet is included in the scanned image until the displaycontrol unit 46 b determines that the paper sheet is included in thescanned image (Step S104). Here, the display control unit 46 brecognizes the paper sheet itself within the scanned image by an imagerecognition technology, to thereby recognize that the paper sheet isincluded in the scanned image. Note that, the user of the mobile device40 can include the paper sheet in the scanned image by adjusting aposition and an orientation of the mobile device 40 so that the papersheet is included in the scanned image displayed on the display unit 42.

When the display control unit 46 b determines in Step S104 that thepaper sheet is included in the scanned image, the display control unit46 b determines whether or not a head position of the paper sheet withinthe scanned image has been designated (Step S105). The user of themobile device 40 can designate the head position by setting the headposition of the paper sheet within the scanned image displayed on thedisplay unit 42 through the operation unit 41.

FIG. 5 illustrates an example of the head position of the paper sheetwithin the scanned image displayed on the display unit 42.

In FIG. 5, a scanned image 50 displayed on the display unit 42 includesa paper sheet 51. In the paper sheet 51 within the scanned image 50, ahead position 60 designated by the user is indicated by beinghighlighted by a thick line.

As illustrated in FIG. 4, when determining in Step S105 that the headposition has not been designated, the display control unit 46 b returnsto the processing of Step S104 again.

When the display control unit 46 b determines in Step S105 that the headposition has been designated, the display control unit 46 b acquires asize of the paper sheet within the scanned image with reference to thehead position designated in Step S105 (Step S106). Specifically, thedisplay control unit 46 b acquires a vertical length of the paper sheetwithin the scanned image and a horizontal length of the paper sheetwithin the scanned image.

Subsequently, the display control unit 46 b calculates the size of theprinting image within the scanned image based on the size acquired inStep S106 and a magnification of the printing image used when theprinting image is printed on the paper sheet (hereinafter referred to as“printing magnification”) (Step S107). For example, in a case where thevertical length and the horizontal length of the paper sheet within thescanned image are 52 and 37, respectively, the display control unit 46 bcalculates a vertical length and a horizontal length of the printingimage within the scanned image as 52 and 37, respectively, when theprinting magnification is 1, and calculates the vertical length and thehorizontal length of the printing image within the scanned image as 104and 74, respectively, when the printing magnification is 2. Note that,an initial value of the printing magnification is, for example, “1”.

Subsequently, the display control unit 46 b generates the printing imagefor a preview image (hereinafter referred to as “preview of the printingimage”) with the size calculated in Step S107 based on the printingimage acquired in Step S101 and the size calculated in Step S107 (StepS108). Here, the display control unit 46 b changes an appearance of thepreview of the printing image based on an output characteristic of theMFP received in Step S102. For example, when the MFP received in StepS102 is a dedicated monochrome machine, the display control unit 46 bchanges the preview of the printing image to a monochrome image.Further, even when the MFP received in Step S102 is a color machine, thedisplay control unit 46 b may change the preview of the printing imagebased on color reproducibility of the MFP.

Subsequently, the display control unit 46 b generates a composite imagethat combines the preview of the printing image generated in Step S108with the scanned image, in a position corresponding in the paper sheetwithin the scanned image to the position of an origin of the printingimage with respect to the paper sheet used when the printing image isprinted on the paper sheet (hereinafter referred to as “printingposition”) (Step S109). Note that, an initial value of the printingposition is, for example, the position of a left end of a top edge ofthe paper sheet when viewed with the head position placed at the top.

Subsequently, the display control unit 46 b displays the composite imagegenerated in Step S109, in other words, the preview image on the displayunit 42 (Step S110). In other words, for example, the display controlunit 46 b displays such a preview image as illustrated in FIG. 6 on thedisplay unit 42 by an augmented reality (AR) technology.

A preview image 70 as illustrated in FIG. 6 is a composite image thatcombines a preview of the printing image 71 with the scanned image 50.

As illustrated in FIG. 4, the display control unit 46 b determineswhether or not the magnification of the preview of the printing imagehas been changed through the operation unit 41 after the processing ofStep S110 (Step S111). For example, the user of the mobile device 40 canchange the magnification of the preview of the printing image bydirectly operating the preview of the printing image displayed on thedisplay unit 42 through the operation unit 41, and can also change themagnification of the preview of the printing image by specificallydesignating a numerical value indicating the magnification through theoperation unit 41.

When the display control unit 46 b determines in Step S111 that themagnification of the preview of the printing image has not been changed,the display control unit 46 b determines whether or not the position ofthe preview of the printing image with respect to the paper sheet withinthe scanned image has been changed through the operation unit 41 (StepS112). For example, the user of the mobile device 40 can change theposition of the preview of the printing image by directly operating thepreview of the printing image displayed on the display unit 42 throughthe operation unit 41, and can also change the position of the previewof the printing image by specifically designating a numerical valueindicating the position through the operation unit 41.

When the display control unit 46 b determines in Step S112 that theposition of the preview of the printing image with respect to the papersheet within the scanned image has not been changed, the display controlunit 46 b returns to the processing of Step S104 again.

When the display control unit 46 b determines in Step S111 that themagnification of the preview of the printing image has been changed, thedisplay control unit 46 b changes the printing magnification based onthe change in the magnification of the preview of the printing image(Step S113), and returns to the processing of Step S104 again. In otherwords, the display control unit 46 b displays, for example, such apreview image as illustrated in FIG. 7 on the display unit 42 by the ARtechnology.

In the preview image 70 illustrated in FIG. 7, the preview of theprinting image 71 has a size of ½ of that of the preview image 70illustrated in FIG. 6. In other words, in a state illustrated in FIG. 7,the printing magnification is set to ½.

As illustrated in FIG. 4, when determining in Step S112 that theposition of the preview of the printing image with respect to the papersheet within the scanned image has been changed, the display controlunit 46 b changes the printing position based on the change in theposition of the preview of the printing image (Step S114), and returnsto the processing of Step S104 again. In other words, the displaycontrol unit 46 b displays, for example, such a preview image asillustrated in FIG. 8 on the display unit 42 by the AR technology.

In the preview image 70 illustrated in FIG. 8, the position of thepreview of the printing image 71 with respect to the paper sheet 51within the scanned image 50 differs from that of the preview image 70illustrated in FIG. 7.

Note that, in the above description, the vertical and horizontalorientations of the paper sheet within the scanned image are the same asthe vertical and horizontal orientations of the scanned image, but, asillustrated in FIG. 9, the vertical and horizontal orientations of thepaper sheet within the scanned image may differ from the vertical andhorizontal orientations of the scanned image.

Further, even when a shape of the paper sheet within the scanned imagehas changed from a rectangle, the display control unit 46 b may changethe shape of the printing image within the composite image based on thechange. For example, even when the shape of the paper sheet within thescanned image has changed to the shape illustrated in FIG. 10, thedisplay control unit 46 b can change the shape of the printing imagewithin the composite image based on the shape of the paper sheet withinthe scanned image as illustrated in FIG. 10. According to thisconfiguration, the mobile device 40 can allow the user to confirm notonly the preview image viewed from the front of the paper sheet but alsothe preview image viewed from various angles with respect to the papersheet by being changed in a photographing angle of the imaging unit 43with respect to the paper sheet.

Note that, the above description is directed to the processing performedwhen the size of the printing image is the same as the size of the papersheet on which the printing image is to be printed. The display controlunit 46 b can acquire the size of the printing image from the printingimage itself, and can therefore generate, even when the size of theprinting image differs from the size of the paper sheet, the preview ofthe printing image to have a suitable size based on the size of theprinting image and the size of the paper sheet in a manner that the sizeof the paper sheet is specifically designated through the operation unit41.

Next, a description is made of an operation of the image forming system10 performed when the MFP is used for printing the printing image.

When being instructed to perform printing on the paper sheet based onthe preview image displayed on the display unit 42 through the operationunit 41, the control unit 46 of the mobile device 40 transmits theprinting image acquired in Step S101 to the MFP received in Step S102along with the printing magnification and the printing position throughthe network communication unit 44.

Therefore, the control unit of the MFP prints the printing imagetransmitted from the mobile device 40 on the paper sheet by a printer byusing the printing magnification and the printing position transmittedfrom the mobile device 40 along with the printing image.

As described above, the mobile device 40 displays the composite imagethat combines the printing image on the paper sheet within the imagescanned by the imaging unit 43, on the display unit 42 as the previewimage (Step S110), and hence an appearance of actual paper sheet can bereflected on the preview image, with the result that the preview imagesimilar to an actual resulting printed image can be generated.

The mobile device 40 recognizes the paper sheet itself within thescanned image by the image recognition technology, to thereby recognizethat the paper sheet is included in the scanned image (Step S104), andhence the composite image that combines the printing image on the papersheet within the scanned image, can be displayed on the display unit 42as the preview image even when a special marker used for recognizingthat the paper sheet is included in the scanned image is not formed onthe paper sheet.

Note that, the mobile device 40 may be configured to recognize that thepaper sheet is included in the scanned image by reading the specialmarker used for recognizing that the paper sheet is included in thescanned image from above the paper sheet.

The mobile device 40 changes the appearance of the printing image withinthe composite image based on the output characteristic of the MFP byreceiving the designation of the MFP that is to print the printing imageon the paper sheet, and hence the preview image suitable for the MFPused for printing the printing image can be easily displayed on thedisplay device.

Note that, the image forming system 10 includes a mobile device as theelectronic device according to the embodiment of the present disclosure,but may include another electronic device such as a PC or the likeinstead of the mobile device.

Further, the image forming system 10 includes an MFP as the imageforming apparatus according to the embodiment of the present disclosure,but the image forming system 10 may include another image formingapparatus such as a dedicated printer, a dedicated copier, a dedicatedfacsimile device, or the like instead of the MFP.

Now, another embodiment of the present disclosure is described withreference to the accompanying drawings.

FIG. 11 illustrates a configuration of an image forming system 110according to the embodiment.

As illustrated in FIG. 11, the image forming system 110 includes aplurality of multifunction peripherals (MFPs) including a MFP 20, amanagement server 30 such as a personal computer (PC) that manages theplurality of MFPs, and a mobile device 140 such as a smartphone. Theplurality of MFPs, the management server 30, and the mobile device 140of the image forming system 110 are communicably connected to oneanother through a network 11 such as a local area network (LAN) or theInternet. The mobile device 140 constitutes an electronic deviceaccording to the present disclosure.

As described above, the MFP 20 has the configuration illustrated in FIG.2. The configuration of each of the plurality of MFPs of the imageforming system 110 is the same as the configuration of the MFP 20.Therefore, the MFP 20 is described below as a representative of theplurality of MFPs of the image forming system 110.

FIG. 12 illustrates a configuration of the mobile device 140.

As illustrated in FIG. 12, the mobile device 140 includes an operationunit 141 serving as an input device with, for example, buttons throughwhich various operations are input, a display unit 142 serving as adisplay device such as an LCD that displays various kinds ofinformation, an imaging unit 143 serving as an imaging device thatphotographs a still image or a moving image, a network communicationunit 144 that communicates to and/or from an external apparatus throughthe network 11 illustrated in FIG. 11, a storage unit 145 including anonvolatile storage device such as an HDD that stores programs andvarious kinds of data, and a control unit 146 that controls the entiremobile device 140.

The operation unit 141 may include an input device including a touchpanel together with the display unit 142.

The storage unit 145 stores a print preview program 145 a executed bythe control unit 146. The print preview program 145 a may be installedon the mobile device 140 at a manufacturing stage of the mobile device140, may be installed on the mobile device 140 additionally from astorage medium such as a universal serial bus (USB) memory, or may beinstalled on the mobile device 140 additionally through the network 11.

The control unit 146 includes, for example, a CPU, a ROM that storesprograms and various kinds of data, and a RAM used as a work area forthe CPU. The CPU executes a program stored in the ROM or the storageunit 145.

The control unit 146 executes the print preview program 145 a stored inthe storage unit 145, to thereby function as a rotational directiondetection unit 146 a, a binding orientation setting unit 146 b, adisplay control unit 146 c, and a printing image acquisition unit 146 d.The rotational direction detection unit 146 a detects, when the papersheet is turned over, a rotational direction of the turning-over withrespect to the paper sheet. The binding orientation setting unit 146 bsets a binding orientation for double-sided printing performed on theMFP. The display control unit 146 c displays the image scanned by theimaging unit 143 on the display unit 142. The printing image acquisitionunit 146 d acquires the printing image to be printed by the MFP.

Next, a description is made of an operation of the image forming system110.

First, a description is made of the operation of the image formingsystem 110 performed when the binding orientation is set for thedouble-sided printing by reading images from a plurality of pages oforiginal documents.

In response to an instruction issued through the operation unit 21, thecontrol unit 28 of the MFP 20 reads images from a plurality of pages oforiginal documents by the scanner 23 to generate a plurality of pages ofprinting images, and transmits the generated plurality of pages ofprinting images to the management server 30 through the networkcommunication unit 26.

Therefore, the management server 30 receives and stores the plurality ofpages of printing images transmitted from the MFP 20.

Next, a description is made of an operation of the mobile device 140performed when the binding orientation is set for the double-sidedprinting performed on the MFP.

The user of the mobile device 140 may activate the print preview program145 a when it is desired to set the binding orientation for thedouble-sided printing performed on the MFP, to thereby cause the controlunit 146 of the mobile device 140 to execute operations illustrated inFIG. 13 to FIG. 15.

FIG. 13 illustrates an operation of the mobile device 140 performed whenthe binding orientation is set for the double-sided printing performedon the MFP. FIG. 14 illustrates an operation of the mobile device 140continued from FIG. 13. FIG. 15 illustrates an operation of the mobiledevice 140 continued from FIG. 13 that is different from the operationillustrated in FIG. 14.

As illustrated in FIG. 13 to FIG. 15, the printing image acquisitionunit 146 d of the mobile device 140 acquires a plurality of pages ofprinting images to be printed on the paper sheet by the MFP through thenetwork communication unit 144 (Step S1101). Here, the printing imageacquisition unit 146 d may acquire from the management server 30 aplurality of pages of printing images generated from a plurality ofpages of original documents by the MFP and stored by the managementserver 30, or may acquire a plurality of pages of printing images ofanother kind. For example, the printing image acquisition unit 146 d mayacquire a plurality of pages of printing images generated from aplurality of pages of original documents by the MFP 20 and transmitteddirectly from the MFP 20 to the mobile device 140, or may acquire aplurality of pages of printing images generated by the PC (not shown)and transmitted to the mobile device 140.

Subsequently, the control unit 146 of the mobile device 140 displays onthe display unit 142 the MFP designation screen used for receiving thedesignation of the MFP to be used for printing the printing images, tothereby receive the designation of the MFP through the operation unit141 (Step S1102).

Subsequently, the display control unit 146 c of the mobile device 140displays the image scanned by the imaging unit 143 on the display unit142 (Step S1103).

Subsequently, the display control unit 146 c determines whether or notthe paper sheet is included in the scanned image until the displaycontrol unit 146 b determines that the paper sheet is included in thescanned image (Step S1104). Here, the display control unit 146 crecognizes the paper sheet itself within the scanned image by the imagerecognition technology, to thereby recognize that the paper sheet isincluded in the scanned image. Note that, the user of the mobile device140 can include the paper sheet in the scanned image by adjusting theposition and the orientation of the mobile device 140 so that the papersheet is included in the scanned image displayed on the display unit142.

When the display control unit 146 b determines in Step S1104 that thepaper sheet is included in the scanned image, the display control unit146 c determines whether or not the head position of the paper sheetwithin the scanned image has been designated (Step S1105). The user ofthe mobile device 140 can designate the head position by setting thehead position of the paper sheet within such a scanned image asillustrated in FIG. 5 displayed on the display unit 142 through theoperation unit 141.

As illustrated in FIG. 13 to FIG. 15, the display control unit 146 cexecutes the processing of Step S1104 when determining in Step S1105that the head position has not been designated.

When the display control unit 146 b determines in Step S1105 that thehead position has been designated, the display control unit 146 c sets,as a target, the printing image on the first page among the plurality ofpages of printing images acquired in Step S1101 (Step S1106).

Subsequently, the display control unit 146 c executes composite imagedisplaying processing illustrated in FIG. 16 (Step S1107).

FIG. 16 illustrates the composite image displaying processing includedin the operations illustrated in FIG. 13 to FIG. 15.

As illustrated in FIG. 16, the display control unit 146 c acquires thesize of the paper sheet within the scanned image with reference to thedesignated head position (Step S1201). Specifically, the display controlunit 146 c acquires the vertical length (length of the left side or theright side of the paper sheet when viewed with the head position placedat the top) of the paper sheet within the scanned image and thehorizontal length (length of the top side or the bottom side when viewedwith the head position placed at the top) of the paper sheet within thescanned image.

Subsequently, the display control unit 146 c generates a printing imagefor a preview image (hereinafter referred to as “preview of the printingimage”) with the size acquired in Step S1201 for the printing image of atarget page based on the printing image of the target page and the sizeacquired in Step S1201 (Step S1202). Here, the display control unit 146c changes the appearance of the preview of the printing image based onthe output characteristic of the MFP received in Step S1102. Forexample, when the MFP received in Step S1102 is a dedicated monochromemachine, the display control unit 146 c changes the preview of theprinting image to a monochrome image. Further, even when the MFPreceived in Step S1102 is a color machine, the display control unit 146c may change the preview of the printing image based on the colorreproducibility of the MFP.

Subsequently, the display control unit 146 c generates a composite imagethat combines the preview of the printing image generated in Step S1202with the scanned image (Step S1203). At this time, the display controlunit 146 c generates the composite image by combining the preview of theprinting image in a specific position in the paper sheet within thescanned image. Here, the specific position in the paper sheet within thescanned image corresponds to a position of a reference point forplacement of the printing image with respect to the paper sheet usedwhen the printing image is printed on the paper sheet. The position ofthe reference point for the placement of the printing image with respectto the paper sheet is, for example, the position of the left end of thetop edge of the paper sheet when viewed with the head position placed atthe top. At this time, the specific position in the paper sheet withinthe scanned image is the position of the left end of the top edge of thepaper sheet within the scanned image when viewed with the head positionplaced at the top.

Subsequently, the display control unit 146 c displays the compositeimage generated in Step S1203. In other words, the display control unit146 c displays the preview image on the display unit 142 (Step S1204)and brings the composite image displaying processing illustrated in FIG.16 to an end. In other words, for example, the display control unit 146c displays such a preview image as illustrated in FIG. 17 on the displayunit 142 by the AR technology.

The preview image 70 illustrated in FIG. 17 is the composite image thatcombines the preview of the printing image 71 with the scanned image 50.The preview image 70 is an example of the composite image generated bycombining the preview of the printing image 71 with the scanned image 50with the left end of the top edge of the preview of the printing image71 coinciding with the left end of the top edge of the paper sheet 51within the scanned image 50 when viewed with the head position 60 placedat the top in a case where the position of the reference point for theplacement of the printing image with respect to the paper sheet is theleft end of the top edge of the paper sheet.

In the above description, the vertical and horizontal orientations ofthe paper sheet 51 within the scanned image 50 are the same as thevertical and horizontal orientations of the scanned image 50, but, asillustrated in FIG. 18, the vertical and horizontal orientations of thepaper sheet 51 within the scanned image 50 may differ from the verticaland horizontal orientations of the scanned image 50. Even at this time,the display control unit 146 c acquires the vertical and horizontalsizes of the paper sheet 51 within the scanned image 50 with referenceto the designated head position 60, and generates the composite image bycombining the preview of the printing image 71 in the specific positionin the paper sheet 51 within the scanned image 50.

Further, even when the shape of the paper sheet 51 within the scannedimage 50 has changed from a rectangle due to an influence of thephotographing position, the display control unit 146 c may change theshape of the preview of the printing image within the composite imagebased on the change. For example, even when the shape of the paper sheet51 within the scanned image 50 has changed to the shape illustrated inFIG. 19, the display control unit 146 c can change the shape of thepreview of the printing image within the composite image based on theshape of the paper sheet 51 within the scanned image 50 as illustratedin FIG. 19. At this time, the display control unit 146 c acquires thelengths of the top side, the bottom side, the left side, and the rightside of the paper sheet 51 within the scanned image 50 when viewed withthe head position 60 placed at the top in Step S1201 illustrated in FIG.16, and changes the shape of the preview of the printing image 71 basedon a ratio of the lengths of the respective sides.

According to this configuration, the mobile device 140 can allow theuser to confirm not only the preview image 70 viewed from the front ofthe paper sheet but also the preview image 70 viewed from various angleswith respect to the paper sheet by being changed in the photographingangle of the imaging unit 143 with respect to the paper sheet.

Note that, the above description is directed to the processing performedwhen the size of the printing image is the same as the size of the papersheet on which the printing image is to be printed. The display controlunit 146 c can acquire the size of the printing image from the printingimage itself. Accordingly, even when the size of the printing imagediffers from the size of the paper sheet, the display control unit 146 ccan generate the preview of the printing image to have a suitable sizebased on the size of the printing image and the size of the paper sheetin a manner that the size of the paper sheet is specifically designatedthrough the operation unit 141.

As illustrated in FIG. 13 to FIG. 15, the control unit 146 determineswhether or not the double-sided printing has been designated after theprocessing of Step S1107 (Step S1108). The user of the mobile device 140can designate the double-sided printing through the operation unit 141.

When the control unit 146 determines in Step S1108 that the double-sidedprinting has not been designated, the control unit 146 executes theprocessing of Step S1104.

When the control unit 146 determines in Step S1108 that the double-sidedprinting has been designated, the control unit 146 sets the double-sidedprinting (Step S1109).

Subsequently, the display control unit 146 c determines whether or notthe paper sheet is included in the image scanned by the imaging unit 143(Step S1110).

When the control unit 146 determines in Step S1110 that the paper sheetis not included in the scanned image, the control unit 146 cancels thesetting of the double-sided printing (Step S1111).

Subsequently, the display control unit 146 c executes the processing ofStep S1104.

When the rotational direction detection unit 146 a of the control unit146 determines in Step S1110 that the paper sheet is included in thescanned image, the rotational direction detection unit 146 a determineswhether or not the paper sheet included in the scanned image has beenturned over based on image processing for the scanned image (StepS1112). The user of the mobile device 140 can, for example, turn overthe paper sheet by manually reversing the paper sheet with his/her hand.

When the rotational direction detection unit 146 a determines in StepS1112 that the paper sheet has not been turned over, the display controlunit 146 c executes the processing of Step S1110.

When the rotational direction detection unit 146 a determines in StepS1112 that the paper sheet has been turned over, the rotationaldirection detection unit 146 a detects the rotational direction of theturning-over with respect to the paper sheet included in the scannedimage based on the image processing for the scanned image (Step S1113).

Subsequently, the binding orientation setting unit 146 b of the controlunit 146 sets the binding orientation for the double-sided printingperformed on the MFP based on the rotational direction detected in StepS1113 (Step S1114). For example, the binding orientation setting unit146 b (i) sets the binding orientation to “top bind” when the rotationaldirection detected in Step S1113 is a rotational direction for turningover the paper sheet about a top edge of the paper sheet, (ii) sets thebinding orientation to “bottom bind” when the rotational directiondetected in Step S1113 is a rotational direction for turning over thepaper sheet about a bottom edge of the paper sheet, (iii) sets thebinding orientation to “left bind” when the rotational directiondetected in Step S1113 is a rotational direction for turning over thepaper sheet about a left edge of the paper sheet, and (iv) sets thebinding orientation to “right bind” when the rotational directiondetected in Step S1113 is a rotational direction for turning over thepaper sheet about a right edge of the paper sheet.

Subsequently, the display control unit 146 c sets as the target theprinting image on the page next to the current target page among theplurality of pages of printing images acquired in Step S1101 (StepS1115).

Subsequently, the display control unit 146 c executes the compositeimage displaying processing illustrated in FIG. 16 (Step S1116).

The preview image 70 displayed on the display unit 142 in Step S1116 is,for example, an image illustrated in FIG. 20A and FIG. 20B. FIG. 20Aillustrates an example of the preview image 70 of the next page obtainedwhen the binding orientation is set to “left bind”. As illustrated inFIG. 20A, the preview of the printing image 71 is displayed by beingcombined with the paper sheet 51 within the scanned image 50 when viewedwith the head position 60 placed at the top. At this time, an apparentbinding position is the right edge of the paper sheet 51. FIG. 20Billustrates an example of the preview image 70 of the next page obtainedwhen the binding orientation is set to “top bind”. As illustrated inFIG. 20B, the preview of the printing image 71 is displayed by beingcombined with the paper sheet 51 within the scanned image 50 when viewedwith the head position 60 of the previous page placed at the bottom. Thehead position 60 of the previous page is displayed at the top of thepaper sheet 51 in the preview image of the previous page, but after theactual paper sheet is turned over about the top edge, displayed at thebottom of the paper sheet 51 in the preview image 70 of the next page.In FIG. 20B, the apparent binding position is the bottom edge of thepaper sheet 51. The head position 60 illustrated in each of FIG. 20A andFIG. 20B is the designated head position, and is the head position ofthe previous page.

As illustrated in FIG. 13 to FIG. 15, the control unit 146 determinesafter Step S1116 whether or not all the plurality of pages of printingimages acquired in Step S1101 have been set as the target (Step S1117).

When the control unit 146 determines in Step S1117 that not all thepages have been set as the target, in other words, when there exists apage that has not yet been set as the target, the display control unit146 c determines whether or not the paper sheet is included in the imagescanned by the imaging unit 143 (Step S1131).

When the display control unit 146 c determines in Step S1131 that thepaper sheet is not included in the scanned image, the bindingorientation setting unit 146 b cancels the setting of the double-sidedprinting and the setting of the binding orientation (Step S1132).

Subsequently, the display control unit 146 c executes the processing ofStep S1104.

When the display control unit 146 c determines in Step S1131 that thepaper sheet is included in the scanned image, the rotational directiondetection unit 146 a determines whether or not the paper sheet includedin the scanned image has been turned over in the same manner as in theprocessing of Step S1112 (Step S1133).

When the rotational direction detection unit 146 a determines in StepS1133 that the paper sheet has not been turned over, the display controlunit 146 c executes the processing of Step S1131.

When the rotational direction detection unit 146 a determines in StepS1133 that the paper sheet has been turned over, the rotationaldirection detection unit 146 a detects the rotational direction of theturning-over with respect to the paper sheet included in the scannedimage in the same manner as in the processing of Step S1113 (StepS1134).

Subsequently, the binding orientation setting unit 146 b determineswhether or not the rotational direction detected in Step S1134 for thetarget page is a reverse direction to the rotational direction detectedin Step S1113 for the previous page (Step S1135).

When the binding orientation setting unit 146 b determines in Step S1135that the rotational direction detected for the target page is thereverse direction to the rotational direction detected for the previouspage, the binding orientation setting unit 146 b cancels the setting ofthe binding orientation set in the immediately previous Step S1114 (StepS1136).

Subsequently, the display control unit 146 c sets as the target theprinting image on the page previous to the current target page among theplurality of pages of printing images acquired in Step S1101 (StepS1137), and executes the composite image displaying processingillustrated in FIG. 8 (Step S1138).

Subsequently, the display control unit 146 c executes the processing ofStep S1110.

When the binding orientation setting unit 146 b determines in Step S1135that the rotational direction detected for the target page is not thereverse direction to the rotational direction detected for the previouspage, the binding orientation setting unit 146 b determines whether ornot the rotational direction detected in Step S1134 for the target pageis the same direction as the rotational direction detected in Step S1113for the previous page (Step S1139).

When the binding orientation setting unit 146 b determines in Step S1139that the rotational direction detected for the target page is not thesame direction as the rotational direction detected for the previouspage, the display control unit 146 c, then executes the composite imagedisplaying processing illustrated in FIG. 16 (Step S1141), and executesthe processing of Step S1131.

When the binding orientation setting unit 146 b determines in Step S1139that the rotational direction detected for the target page is the samedirection as the rotational direction detected for the previous page,the display control unit 146 c determines whether or not the paper sheetis included in the image scanned by the imaging unit 143 (Step S1142).

When the display control unit 146 c determines in Step S1142 that thepaper sheet is not included in the scanned image, the bindingorientation setting unit 146 b cancels the setting of the double-sidedprinting and the setting of the binding orientation (Step S1143).

Subsequently, the display control unit 146 c executes the processing ofStep S1104.

When the display control unit 146 c determines in Step S1142 that thepaper sheet is included in the scanned image, the display control unit146 c determines whether or not the head position has been designated(Step S1144). The user of the mobile device 140 can designate the headposition by setting the head position of the paper sheet within thescanned image displayed on the display unit 142 through the operationunit 141.

When the display control unit 146 c determines in Step S1144 that thehead position has not been designated, the display control unit 146 cexecutes the processing of Step S1142.

When the display control unit 146 c determines in Step S1144 that thehead position has been designated, the display control unit 146 c setsas the target the printing image on the page next to the current targetpage among the plurality of pages of printing images acquired in StepS1101 (Step S1145), and executes the composite image displayingprocessing illustrated in FIG. 16 (Step S1146).

Subsequently, the control unit 146 determines whether or not all theplurality of pages of printing images acquired in Step S1101 have beenset as the target (Step S1147).

When the control unit 146 determines in Step S1147 that all the pageshave been set as the target, the display control unit 146 c executes theprocessing of Step S1146.

When the control unit 146 determines in Step S1147 that not all thepages have been set as the target, in other words, when there exists apage that has not yet been set as the target, the display control unit146 c executes the processing of Step S1110.

When the control unit 146 determines in Step S1117 that all the pageshave been set as the target, the display control unit 146 c determineswhether or not the paper sheet is included in the image scanned by theimaging unit 143 (Step S1161).

When the display control unit 146 c determines in Step S1161 that thepaper sheet is not included in the scanned image, the bindingorientation setting unit 146 b cancels the setting of the double-sidedprinting and the setting of the binding orientation (Step S1162).

Subsequently, the display control unit 146 c executes the processing ofStep S1104.

When the display control unit 146 c determines in Step S1161 that thepaper sheet is included in the scanned image, the rotational directiondetection unit 146 a determines whether or not the paper sheet includedin the scanned image has been turned over in the same manner as in theprocessing of Step S1112 (Step S1163).

When the rotational direction detection unit 146 a determines in StepS1163 that the paper sheet has not been turned over, the display controlunit 146 c executes the processing of Step S1161.

When the rotational direction detection unit 146 a determines in StepS1163 that the paper sheet has been turned over, the rotationaldirection detection unit 146 a detects the rotational direction of theturning-over with respect to the paper sheet included in the scannedimage in the same manner as in the processing of Step S1113 (StepS1164).

Subsequently, the binding orientation setting unit 146 b determineswhether or not the rotational direction detected in Step S1164 for thetarget page is a reverse direction to the rotational direction detectedin Step S1113 for the previous page (Step S1165).

When the binding orientation setting unit 146 b determines in Step S1165that the rotational direction detected for the target page is thereverse direction to the rotational direction detected for the previouspage, the binding orientation setting unit 146 b cancels the setting ofthe binding orientation set in the immediately previous Step S1114 (StepS1166).

Subsequently, the display control unit 146 c sets as the target theprinting image on the page previous to the current target page among theplurality of pages of printing images acquired in Step S1101 (StepS1167), and executes the composite image displaying processingillustrated in FIG. 16 (Step S1168).

Subsequently, the display control unit 146 c executes the processing ofStep S1110.

When the binding orientation setting unit 146 b determines in Step S1165that the rotational direction detected for the target page is not thereverse direction to the rotational direction detected for the previouspage, the display control unit 146 c displays the error indicating thatthe direction of the turning-over is inappropriate on the display unit142 (Step S1169), then executes the composite image displayingprocessing illustrated in FIG. 16 (Step S1170), and executes theprocessing of Step S1161.

Next, a description is made of an operation of the image forming system110 performed when the MFP is used for print the printing image.

When instructed to perform printing on the paper sheet through theoperation unit 141, the control unit 146 of the mobile device 140transmits the plurality of pages of printing images acquired in StepS1101 along with double-sided printing setting information indicating asetting relating to the double-sided printing to the MFP received inStep S1102 through the network communication unit 144. Here, thedouble-sided printing setting information includes informationindicating whether or not the double-sided printing has been set in StepS1109. Further, the double-sided printing setting information includesinformation on the binding orientation of each paper sheet set in StepS1114 when the double-sided printing is set.

Therefore, the control unit of the MFP uses the double-sided printingsetting information transmitted from the mobile device 140 along withthe printing image to print the printing image transmitted from themobile device 140 on the paper sheet by the printer. In other words,when the double-sided printing setting information includes theinformation indicating that the double-sided printing has not been set,the control unit of the MFP subjects the printing image transmitted fromthe mobile device 140 to single-sided printing on the paper sheet. Onthe other hand, when the double-sided printing setting informationincludes the information indicating that the double-sided printing hasbeen set, the control unit of the MFP subjects the printing imagetransmitted from the mobile device 140 to the double-sided printing onthe paper sheet based on the information on the setting of the bindingorientation of each paper sheet included in the double-sided printingsetting information.

As described above, the mobile device 140 causes the user to turn overthe actual paper sheet, to thereby set the binding orientationcorresponding to the rotational direction of the turning-over withrespect to the paper sheet (Steps S1112 to S1114), and hence it ispossible to allow the user to intuitively designate the bindingorientation for the double-sided printing performed on the MFP.

The mobile device 140 causes the user to turn over the actual papersheet to set the binding orientation (Step S1114), and then causes theuser to actually turn over the paper sheet in a rotational direction forcancellation serving as a rotational direction used for canceling thesetting, to thereby cancel the setting of the binding orientation (StepsS1133 to S1136 or Steps S1163 to S1166) that allows the user to easilyinstruct the cancellation of the setting of the binding orientation.

Here, the mobile device 140 causes the user to turn over the actualpaper sheet to set the binding orientation, and then causes the user toactually turn over the paper sheet in the rotational direction forcancellation that is the reverse direction to the rotational directionof the turning-over, in other words, causes the user to actually executean operation that is reverse to the operation for setting the bindingorientation (YES in Step S1135 or YES in Step S1165), to thereby cancelthe setting of the binding orientation (Steps S1133 to S1136 or StepsS1163 to S1166) that allows the user to intuitively instruct thecancellation of the setting of the binding orientation.

Note that, the mobile device 140 may be provided with the rotationaldirection for cancellation that is a rotational direction other than thereverse direction to the rotational direction for setting the bindingorientation. Further, the mobile device 140 may not be provided with therotational direction for cancellation itself.

The mobile device 140 sets the binding orientation for the double-sidedprinting on the first paper sheet of two successive paper sheets to besubjected to the double-sided printing (Step S1114), and when arotational direction for shift that is a rotational direction used forshifting to a state of being ready to receive the setting of the bindingorientation for the double-sided printing on the second paper sheet isdetected (YES in Step S1139), shifts to the state of being ready toreceive the setting of the binding orientation for the double-sidedprinting on the second paper sheet (YES in Step S1110 and Step S1112).According to this configuration, the mobile device 140 causes the userto turn over the actual paper sheet to set the binding orientation, andthen causes the user to actually turn over the paper sheet in therotational direction for shift, to thereby shift to the state of beingready to receive the setting of the binding orientation for thedouble-sided printing on the second paper sheet of the two successivepaper sheets to be subjected to the double-sided printing that allowsthe user to easily instruct the shift to the state of being ready toreceive the setting of the binding orientation for the double-sidedprinting on the second paper sheet. Note that “a state of being ready”may be herein referred to as “a ready state.”

Here, the mobile device 140 causes the user to turn over the actualpaper sheet to set the binding orientation, and then causes the user toactually turn over the paper sheet in the rotational direction for shiftthat is the same direction as the rotational direction of theturning-over (YES in Step S1139), to thereby be able to shift to thestate of being ready to receive the setting of the binding orientationfor the double-sided printing on the second paper sheet (YES in StepS1110 and Step S1112). However, the mobile device 140 may be providedwith the rotational direction for shift that is a rotational directionother than the same direction as the rotational direction for settingthe binding orientation. For example, the mobile device 140 may beconfigured to employ all rotational directions other than the rotationaldirection for cancellation as the rotational direction for shift.Further, the mobile device 140 may not be provided with the rotationaldirection for shift itself when, for example, being configured to shiftto the state of being ready to receive the setting of the bindingorientation for the double-sided printing on the second paper sheetbased on a specific operation performed through the operation unit 141or being configured in the first place so that all the settings of thebinding orientations for the double-sided printing on a plurality ofsuccessive paper sheets are the same.

The mobile device 140 displays the composite image that combines theprinting image on the paper sheet within the image scanned by theimaging unit 143, on the display unit 142 as the preview image (StepS1204), and can therefore reflect the appearance of the actual papersheet on the preview image, with the result that the preview image closeto the actual printed matter can be generated.

The mobile device 140 recognizes the paper sheet itself within thescanned image by the image recognition technology, to thereby recognizethat the paper sheet is included in the scanned image, and can thereforedisplay the composite image that combines the printing image on thepaper sheet within the scanned image, on the display unit 142 as thepreview image even when the special marker used for recognizing that thepaper sheet is included in the scanned image is not formed on the papersheet.

Note that, the mobile device 140 may be configured to recognize that thepaper sheet is included in the scanned image by reading the specialmarker used for recognizing that the paper sheet is included in thescanned image from above the paper sheet.

The mobile device 140 changes the appearance of the preview of theprinting image within the composite image based on the outputcharacteristic of the MFP by receiving the designation of the MFP thatis to print the printing image on the paper sheet, and hence the previewimage suitable for the MFP used for printing the printing image can beeasily displayed on the display unit 142.

The mobile device 140 displays the composite image that combines theprinting image on the paper sheet within the image scanned by theimaging unit 143, on the display unit 142 as the preview image (StepS1204), and can therefore allow the user to specifically imagine aresult of the double-sided printing, with the result that it is possibleto allow the user to more intuitively designate the binding orientationfor the double-sided printing performed on the MFP. However, the mobiledevice 140 may be configured to inhibit the preview image from beingdisplayed on the display unit 142 when the binding orientation for thedouble-sided printing is set in the processing illustrated in FIG. 13 toFIG. 15.

The mobile device 140 can allow the user to confirm the scanned imagedisplayed on the display unit 142 when causing the user to turn over theactual paper sheet, and can therefore allow the user to confirm that anact of turning over the paper sheet is recognized by the mobile device140 that can improve operability. However, the mobile device 140 may beconfigured to inhibit the image scanned by the imaging unit 143 frombeing displayed on the display unit 142 when the binding orientation forthe double-sided printing is set in the processing illustrated in FIG.13 to FIG. 15.

Note that, the image forming system 110 includes a mobile device as theelectronic device according to the embodiment of the present disclosure,but may include another electronic device such as a PC or the likeinstead of the mobile device.

Further, the image forming system 110 includes an MFP as the imageforming apparatus according to the embodiment of the present disclosure,but may include another image forming apparatus such as a dedicatedprinter, a dedicated copier, a dedicated facsimile device, or the likeinstead of the MFP.

It should be understood that various changes and modifications to thepresently embodiments described herein will be apparent to those skilledin the art. Such changes and modifications can be made without departingfrom the spirit and scope of the present subject matter and withoutdiminishing its intended advantages. It is therefore intended that suchchanges and modifications be covered by the appended claims.

What is claimed is:
 1. A print preview method, comprising: scanning, viaan imaging device, a paper sheet, wherein the paper sheet comprises afirst side and a second side, wherein scanning the paper sheet producesan image; acquiring, via a printing image acquisition unit, a printingimage; printing, via an image forming apparatus, the printing image;controlling, via a display control unit, a display device to display, ifthe produced image includes an image of a paper sheet, a composite imagein which the acquired printing image and the image of the paper sheetincluded in the produced image are combined; detecting, via a rotationaldirection detection unit, if the paper sheet is turned over from thefirst side to the second side, a rotational direction in which the papersheet is turned over with respect to the paper sheet; and setting, via abinding orientation setting unit, a binding orientation for double-sidedprinting performed on the image forming apparatus, wherein: therotational direction detection unit detects, if the produced imageincludes the image of the paper sheet, the rotational direction withrespect to the paper sheet; and the binding orientation setting unitsets the binding orientation corresponding to the detected rotationaldirection.
 2. The print preview method according to claim 1, furthercomprising: determining, via the display control unit, whether the imageof the paper sheet is included in the produced image using an imagerecognition technology.
 3. The print preview method according to claim1, further comprising: receiving, via an apparatus designation receptionunit, a designation indicative of the image forming apparatus; andchanging, via the display control unit, an appearance of the acquiredprinting image within the composite image based on an outputcharacteristic of the designated image forming apparatus.
 4. The printpreview method according to claim 1, further comprising: canceling, viathe binding orientation setting unit, after setting the bindingorientation, the binding orientation for double-sided printing if arotational direction for cancellation is detected via the rotationaldirection detection unit.
 5. The print preview method according to claim4, wherein the rotational direction for cancellation includes a reversedirection to the rotational direction if the binding orientation settingunit cancels the setting corresponding to the rotational direction. 6.The print preview method according to claim 1, further comprising:shifting, via the binding orientation setting unit, to a ready state toreceive the setting of the double-sided printing for a second papersheet, after the setting of the double-sided printing for a first papersheet, if a rotational direction for shifting to the ready state toreceive the setting of the double-sided printing for the second papersheet is detected via the rotational direction detection unit.